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Background: Current guidelines for the conduct of cost-effectiveness analysis (CEA) are mainly applicable to 
facility-based interventions in high-income settings. Differences in the unit of analysis and the high cost of 
data collection can make these guidelines challenging to follow within public health trials in low- and middle- 
income settings. 

Objective: This paper reflects on the challenges experienced within our own work and proposes solutions that 
may be useful to others attempting to collect, analyse, and compare cost data between public health research 
sites in low- and middle- income countries. 

Design: We describe the generally accepted methods (norms) for collecting and analysing cost data in a single- 
site trial from the provider perspective. We then describe our own experience applying these methods within 
eight comparable cluster randomised, controlled, trials. We describe the strategies used to maximise 
adherence to the norm, highlight ways in which we deviated from the norm, and reflect on the learning and 
limitations that resulted. 

Results: When the expenses incurred by a number of small research sites are used to estimate the cost- 
effectiveness of delivering an intervention on a national scale, then deciding which expenses constitute 'start- 
up' costs will be a nontrivial decision that may differ among sites. Similarly, the decision to include or exclude 
research or monitoring and evaluation costs can have a significant impact on the findings. We separated out 
research costs and argued that monitoring and evaluation costs should be reported as part of the total trial 
cost. The human resource constraints that we experienced are also likely to be common to other trials. As we 
did not have an economist in each site, we collaborated with key personnel at each site who were trained to use 
a standardised cost collection tool. This approach both accommodated our resource constraints and served as 
a knowledge sharing and capacity building process within the research teams. 

Conclusions: Given the practical reality of conducting randomised, controlled trials of public health 
interventions in low- and middle- income countries, it is not always possible to adhere to prescribed guidelines 
for the analysis of cost effectiveness. Compromises are frequently required as researchers seek a pragmatic 
balance between rigor and feasibility. There is no single solution to this tension but researchers are 
encouraged to be mindful of the limitations that accompany compromise, whilst being reassured that 
meaningful analyses can still be conducted with the resulting data. 
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Randomised, controlled trials of public health inter- 
ventions are often conducted in more than one 
context to gauge their effectiveness in different 
populations (1, 2). The trials frequently need to establish 



whether the interventions are cost-effective and econom- 
ically feasible at scale (3). Current guidelines for the 
conduct of cost-effectiveness analysis (CEA) are mainly 
applicable to facility-based interventions in high-income 
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settings, where the unit of randomisation is the indi- 
vidual and patient level data is readily available (4, 5). 
These guidelines can be difficult to implement for com- 
plex interventions in resource poor countries, when pro- 
vider cost data are unavailable or of poor quality and the 
unit of randomisation is the cluster or village. Others have 
begun to address the difficulties in adapting these guide- 
lines to the low- and middle-income countries' contexts 
(6). To our knowledge, however, no study has yet con- 
sidered the additional complexity of adapting those guide- 
lines to compare findings among resource poor settings. 
This paper thus aims to enrich the existing literature 
on adapting guidelines for implementation in resource 
poor settings. In addition, this paper aims to extend that 
evidence base to highlight the challenges one might ex- 
perience when it is necessary to compare findings from 
trials conducted in different settings. 

This paper reflects our own experience working within 
community-based, cluster-randomised, controlled trials 
conducted in seven sites, across four low- and middle- 
income countries, between 1999 and 2011. We describe 
the challenges of the costing process, the solutions and 
strategies adopted, and the lessons learned. The inter- 
vention we use as our working example involved women's 
groups practising participatory learning and action to 
improve care-seeking and care practices. A meta-analysis 
demonstrated that women's groups are associated with a 
34% reduction in maternal mortality and a 23% reduction 
in neonatal mortality (7). This paper focuses only on the 
process of cost data collection and analysis for these 
trials and does not discuss the collection and analysis of 
epidemiological data or cost-effectiveness. 

This paper is organised as follows: We first provide 
a detailed summary of the working example. We then 
summarise available guidelines for identifying, collecting, 
and analysing costs, before explaining our difficulties in 
applying the guidelines. In that explanation, we highlight 
the particular challenges that may arise when costs must 
be compared among sites. Finally, we reflect on the key 



lessons learned, emphasising those that may be general to 
other studies of this type. Each section begins with a brief 
review of the literature, followed by reflections on our 
working example, highlighting the challenges encoun- 
tered and describing the solutions adopted. 

A working example: multisite community 
participatory intervention 

This paper draws on experience from eight cluster- 
randomised controlled trials that tested the impact of 
women's groups on mortality. The groups had a four- 
phase participatory learning and action cycle to identify 
and prioritise problems during pregnancy, delivery, and 
postpartum; to plan and implement locally feasible strat- 
egies to address problems; and to evaluate their activities 
(7). The unit of randomisation was the village (or slum- 
cluster in Mumbai). The intervention employed local 
facilitators (were not health workers) to conduct monthly 
or fortnightly meetings. The first trial began in 2001 in 
Makwanpur, Nepal, and seven similar trials were in- 
itiated in other sites between 2005 and 2008. Four of the 
sites adopted a factorial design and tested a second 
intervention alongside the women's groups (Table 1). 
Context-specific health service strengthening was under- 
taken to a varying degree in the intervention and control 
areas in all trials. Further details of the trials and their 
effectiveness have been published elsewhere (8-15). 

Despite similarities in design and implementation, 
there were differences among sites. These differences in- 
cluded the size of the study population, the geography of 
the study area, baseline neonatal and maternal mortality 
rates, and participation rates among pregnant women. 
There were also differences in the characteristics of the 
implementing organisations, some of which were pre- 
viously well-established, while others were set up for 
the purpose of the trial. Such differences can challenge 
standard guidelines for the collection and analysis of data 
for a comparative costing. How then can research teams 
allow for some acceptable level of heterogeneity while 



Table 1. Women's group trials 



Location 


Organisation 


Trial dates (Baseline dates) 


Second intervention 


Cost analysis 


Makwanpur, Nepal 


MIRA 


2001-2003 (1999-2000) 


N/A 


Borghi et al. 2005 


Dhanusha, Nepal 


MIRA 


2007-2011 (2006-2007) 


Sepsis management 


Under progress 


Orissa and Jharkhand, India 


Ekjut 


2005-2008 (2004-2005) 


N/A 


Tripathy et al. 2010 


Mumbai, India 


SNEHA 


2006-2009 (2005-2006) 


N/A 


N/A (see More 2012) 


Bangladesh 


BADAS-PCP 


2005-2007 (2003) 


Resuscitation training 


N/A (see Azad et al. 2010) 


Bangladesh (scale-up) 


BADAS-PCP 


2009-2011 (2008) 


N/A 


Under progress 


Malawi 


MaiKhanda 


2008-2010 (2007-2008) 


Quality improvement 


Under progress 


Mchinji, Malawi 


MaiMwana 


2005-2009 (2005) 


Volunteer peer counselling 


Lewycka et al. 2012 



Sources: Manandhar et al. 2004; Borghi et al. 2005; Tripathy et al. 2010; Azad et al. 2010; Houweling et al. 2011; More et al. 2012; 
Colbourn et al. 2012; Prost et al. 2013. 
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maintaining robust comparability across trials? This was 
the task facing our research team in the context of 
these trials. 

General considerations for cost data 
collection 

Deciding how costs will be collected necessitates a number 
of early decisions. Crucial decisions include study perspec- 
tive, time horizon, how and where data will be sourced, how 
resource use will be measured and valued, and how 
accounting costs will be adjusted to arrive at economic 
costs. Finally, investigators need to consider how to allo- 
cate joint costs, how to separate start-up costs from imple- 
mentation costs, and whether to cost out other trial 
activities such as monitoring and evaluating research, or 
cost only the intervention itself. Each of these decisions 
should be framed within the overall objectives of the study. 
Our experience with this process is described here. 

Perspective: patient, provider, or societal 

A costing can be conducted from the perspective of the 
patient, provider, purchaser, sponsor, government, or 
society. Each of these perspectives will bear different 
costs for the same intervention (16, 17). For example, if a 
costing is conducted from the provider perspective, costs 
incurred by patients would not be relevant. 

In the context of our trials, two perspectives were 
appropriate: provider and societal. The provider perspec- 
tive enables any future provider to weigh the costs and 
benefits (to their institution) of taking on this intervention 
against those of other interventions. The societal perspec- 
tive adds the costs of participation incurred by house- 
holds to the costs incurred by the provider (3). Because 
community participatory interventions may produce non- 
health benefits such as community empowerment (15, 18), 
a societal perspective would potentially capture health 
and non-health costs and benefits to the women and their 
communities, as well as to providers (3, 19). 

For several reasons, we elected to adopt a provider per- 
spective. In practice, our trial sites did not have sufficient 
funds or field experience to collect the household level 
economic data required for a societal perspective. In 
addition, the monitoring and evaluation questionnaires 
were already lengthy, and there was an ethical concern 
about burdening respondents further. In Dhanusha, 
Nepal, field teams complained that the monitoring and 
evaluation questionnaire used in 2010 took a minimum of 
two hours and two to three visits to complete. In this 
situation, we might have conducted a separate 'econom- 
ics' survey among a subset of households. However, our 
surveillance teams were already fully committed, and we 
could not afford to hire and train new teams. Further, we 
were reluctant to subject participating households to yet 
another data collection effort, fearing that this would 



increase the likelihood of women dropping out of the trial 
or refusing to participate in future work. 

Time horizon 

Interventions are typically evaluated for a period of 'full 
scale activity' to assess the desired effects (20). The World 
Health Organization recommends a 10-year time horizon 
to explore the effectiveness of a trial in a single site, and 
Ramsey et al. (21) recommend that a common time 
horizon be used for costs and effects. Dhaliwal et al. (6) 
however, argue that educational programmes can be 
examined over a one- or two-year time frame because, 
in this sector, start-up costs are relatively small. These 
recommendations do not provide specific guidance for 
costing community-based health interventions in multiple 
sites. 

In our case, and consistent with the approach sug- 
gested by Ramsey et al. (21) the costing time horizon was 
determined by the trial duration, that is, two-and-a- 
half to three years. To this, we added a start-up period 
that included activities conducted before the trial began. 
The end point was the same as that used for calculating 
programme effect, which was chosen by the team con- 
ducting the epidemiological analysis. 

Identifying, measuring, and valuing relevant costs 

The literature suggests that cost collection methods de- 
pend on the perspective of the CEA. In general, micro- 
costing tends to be used for studies with a consumer or 
societal perspective (22, 23), while gross-costing can be 
used for a provider perspective (24). 

Micro-costing records resource use at the patient (cost- 
object) level and enumerates overheads and capital costs 
such as office rent and electricity costs separately. These 
costs are added to measure overall resource use and 
the total cost of service provision (25, 26). Studies using 
the micro-costing approach generally source data from 
administrative databases, self-reported activity logs, time 
and motion studies, and manager surveys (25-27). 

When gross-costing, the total cost of service provision 
is first calculated at the institutional level and then 
disaggregated to departments, service units, or patients. 
Resources are generally assumed to be evenly distributed 
across end users. For this reason gross-costing is not 
suitable for services that are not the same for all end users 
(25, 26, 28). Studies using a gross-costing approach fre- 
quently collect retrospective data from accounting data- 
bases, tariff books, market prices, or published studies. 
Because these data are financial or accounting costs, they 
need to be converted into economic costs. Economic 
costs include the value of the next best alternative forgone 
by making a particular choice. In practice, in our sites, 
this meant that although donated goods and volunteer 
time appeared as a zero cost in the accounting data, they 
could not be treated as zero cost items for the purpose 
of the economic costing. Treating such items as zero 
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cost would result in a downward bias in the cost of the 
intervention and an upward bias in the cost-effectiveness 
ratio. This bias is particularly concerning if the goods are 
not available for free in all settings or if the intervention 
cannot be delivered using volunteers at scale or in all 
settings (29). There is no consensus in the literature on 
the 'best' technique for converting accounting costs to 
economic costs, especially in resource poor settings. One 
method is to use market prices (28). However, in such 
settings, market prices may not reflect the true availability 
of labour and capital owing to the presence of formal and 
informal labour and capital markets. Market prices may 
therefore need to be adjusted. A commonly used adjust- 
ment technique is shadow pricing, wherein a good or 
service is assigned a price defined by what an individual 
must give up to gain an extra unit of the good or service 
(29, 30). However, such adjustment techniques require 
context and resource specific calculations, which are time 
and labour intensive. 

We took a provider perspective and collected costs 
retrospectively using a gross-costing approach (9, 11, 31, 
32). Data was sourced from project accounts of the partner 
institutions in the study countries. Almost every site had 
either made use of donated goods or volunteers to imple- 
ment the intervention to some degree. For example, office 
space in Dhaka, Bangladesh, was provided for free by the 
government. This space was valued at private sector rental 
rates. External consultants, who were not paid by the 
project, provided key input into the start-up phase. For 
these consultants, we estimated the number of consultancy 
hours provided and valued this at a rate commensurate 
with their qualifications and work experience. 

In addition to the expected challenges of converting 
accounting to economic costs, we encountered another 
practical limitation of project accounts data not com- 
monly documented in the literature. In Bangladesh, we had 
more than one funder over the full duration of the trial, and 
each funder kept project accounts using their own method 
within their own files. Calculating costs for this trial 
required sourcing cost data from multiple sources includ- 
ing funders no longer involved with the trial. These data 
then had to be reconciled (i.e. significantly reformatted) 
before they could be combined. Some of the data was 
available in hard copy form only and needed to be entered 
into a spread sheet before we could begin the process of 
reconciliation. This process was significantly more com- 
plex and time consuming than we had anticipated and 
resulted in delays to the final analysis of cost-effectiveness 
and to the publication of the main trial paper (10). 

Dealing with joint costs 

Joint costs occur when two or more outputs, services, 
or activities are produced from the same input. Common 
examples include staff who work in both research 
and implementation or who oversee several arms of a 



multi-arm trial. The same could be said for office space 
housing staff from different projects or having different 
functions within a single project. If a resource is used by 
several components of an intervention, or by other 
programmes that are not part of the intervention, the 
cost associated with that resource must be allocated 
proportionally in some way. This is commonly done by 
assessing time or resource use, measured either by ob- 
servation or by self-reported methods (17). 

For each of our research sites, we needed to distinguish 
start-up from running costs, women's groups from health 
service strengthening, our package of interventions from 
other intervention(s) implemented by partner institutions, 
monitoring and evaluation from implementation, research 
from implementation, and process evaluation from im- 
plementation. At all sites, we could directly assign most 
incurred costs to these categories using appropriate labels 
in the project accounts. However, for resources that were 
used across categories, we had little prospective informa- 
tion to use as a basis for the proportional allocation of 
costs. In some cases, we used information from vehicle 
logbooks to allocate vehicle purchase, maintenance, and 
fuel costs as well as drivers' salary costs (MaiMwana trial) 
or used fax machine minutes for fax machine costs 
(Bangladesh trial). Staff time sheets for managerial or 
administrative staff were available for only two of the trials 
(Makwanpur and Ekjut trials) (8, 9). 

We allocated programme costs to components or 
categories using a two-step process. First, we directly 
allocated as many of the staff, material, and capital cost 
items as possible using the methods described above. 
Second, the economics team, in consultation with local 
managerial staff, decided upon a joint cost allocation rule 
to divide the remainder among the programme compo- 
nents. For example, in MaiMwana (Malawi), 25% of 
the joint costs were allocated to women's groups, 25% 
to monitoring and evaluation, 19% to the peer counsel- 
ling intervention, 16% to health service strengthening, 
10% to process evaluation, and 5% to research. In the 
Bangladesh scale-up trial, 40% was allocated to women's 
groups, 40% to monitoring and evaluation, and 10% 
to health service strengthening and process evaluation, 
respectively. Although this resulted in differing alloca- 
tion rules across sites, the decisions reflected the under- 
standing of actual resource use by local staff who had 
worked in the implementing organisation during the trial. 

Start-up and implementation costs 

Separating start-up and implementation costs is essential 
if the analysis must tabulate the cost of scaling up 
an intervention. If, however, an intervention is tested at 
scale, this may not be necessary. The start-up period is 
the time between the decision to implement an interven- 
tion and delivering it to the first beneficiary (33). 
Typically, start-up costs include (but are not limited to) 



4 

(page number not for citation purpose) 



Citation: Glob Health Action 2014, 7: 23257 - http://dx.doi.org/10.3402/gha.v7.23257 



CEA of complex interventions across multiple countries 



costs incurred when recruiting personnel, procuring 
office space and equipment, and training field workers. 
Implementation costs are those incurred when the inter- 
vention is being implemented and commonly include 
salaries, transportation costs, overheads, additional ma- 
terials, and capital goods. 

In Nepal (8), we defined start-up costs by type of 
activity. For example, the design of the pictorial card game 
was classified as a women's group start-up cost, and the 
design of health worker training manuals was classified as 
a health service strengthening start-up cost (31). However, 
the field team found this process time intensive and 
onerous and were reluctant to further participate in the 
costing. As such, we took a more pragmatic approach to 
ensure the on-going cooperation of the field teams. We de- 
fined a start-up period during which all costs incurred 
were classified as start-up. The remainder of the costs were 
classified as implementation costs. In a few instances, 
costs incurred in the implementation period were classi- 
fied as start-up if we considered them to be essential for 
implementation to continue. These included the recruit- 
ment and training of new facilitators to replace facilita- 
tors who dropped out or moved away. Training of health 
staff was part of the health service strengthening provided 
to intervention and control areas and therefore was an 
implementation cost. 

Research and monitoring and evaluation costs 

Once an intervention proves to be effective in a given 
context, scaling up or replicating that intervention will 
seldom require the same research activities required for the 
initial trial. However, some on-going monitoring and 
evaluation of the intervention is desirable. The economic 
evaluation of a trial may want to split research from 
implementation costs and monitoring and evaluation from 
research costs. The study by Gilson et al. (34) is one of the 
few to report that research costs were excluded from the 
cost analysis. We were unable to find any study that 
distinguished monitoring and evaluation costs from re- 
search costs. 

In our trials, we defined monitoring and evaluation as 
all activities conducted by the surveillance team including 
data officers who conducted data entry and cleaning. It 
was relatively straightforward to identify the monitoring 
and evaluation costs because all our sites had a separate 
monitoring and evaluation team. Thus, all monitoring 
and evaluation expenses were a separate line item in the 
project accounts. We defined research costs as activities 
related to the analysis of data, dissemination of results, 
and planning of further interventions. Most research 
activities were conducted by staff from the UK partner 
institution. However at all sites, local staff also engaged 
in research activities. Our task was to identify and deduct 
the latter type of research cost from local project 
accounts. We identified the cost of meetings and travel 



for which the primary purpose was research, retro- 
spectively estimated the proportion of time that indivi- 
dual local staff members spent on research each year, and 
used this proportion to allocate relevant capital costs 
(e.g. laptops) to research. We also assigned a proportion, 
based upon staff time use, of all joint costs to research. 
These research costs were then deducted from the total 
cost of the project, while the monitoring and evaluation 
costs were included in the total cost. 

Owing to the design of randomised controlled trials, 
roughly half of monitoring activities are conducted in 
control areas. It could be argued that this is a research 
cost that should be subtracted from the total programme 
cost. We encountered two main difficulties in doing so. 
The first is that we only had one monitoring and 
evaluation team per site and therefore, could not clearly 
differentiate between the costs incurred in intervention 
versus control areas. Second, the very act of monitoring 
may have had an impact on the effectiveness of the 
intervention. Thus, we did not attempt to identify the 
proportion of monitoring and evaluation activity that 
was a part of research activity. Instead, we calculated 
total monitoring and evaluation cost and reported this 
alongside the intervention implementation cost. 

Complexities of a multisite CEA 

Making cost collection decisions, as described earlier, 
can be challenging in a single setting. Making those de- 
cisions when the results must be comparable across 
multiple settings can be complex in the extreme. In partic- 
ular, we reflect upon our efforts at collaboration and 
standardisation. 

Collaborative efforts in cost identification and 
collection 

Many economic evaluations may be conducted by eco- 
nomists based in the evaluating institution and not by 
staff at trial sites, making a very strong case for collabora- 
tion on the CEA of a multisite, randomised, controlled 
trial. First, collaboration provides opportunities for skill 
sharing and capacity building, enabling the integration 
of economic activities into country teams (1). Second, 
no one economist can be in all sites at all times, and few 
funders will finance one qualified economist per site! 
Third, collaboration facilitates data consistency. 

We found it useful to establish a key 'costing contact' 
at each site to oversee data collection. This ensured that 
costs were captured without creating a significant addi- 
tional burden or without introducing parallel systems 
that replicated data already being collected for other 
purposes. These contacts were not all economists or 
finance personnel but were those interested in the costing 
process and volunteered to take on the role. Our costing 
contacts collected data with the help of a custom-made 
tool described in the next section. Although we found 
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collaboration beneficial overall, we also argue that it was 
important to have a central lead on the CEA who was 
able to support the sites, ensure overall consistency, and, 
ultimately, shape the final analysis. 

Standardisation: designing a cost collection tool 

Standardising economic evaluation methods enables 
users to compare the results of evaluations for different 
interventions or for a similar intervention in different 
settings (35). The key 'standard' aspects that facilitate 
comparative analysis are the perspective, measures of 
outcome or effect, and the definition of inputs and costs 
(35, 36). If the collection of costs is decentralised to 
implementing partners who may have varying levels of 
expertise with economic evaluation, achieving this level 
of standardisation can be challenging. 

During design and implementation of our trials, 
economists were engaged on an ad hoc basis (9, 21). 
This resulted in variable progress toward conducting 
the costing activities across different sites, despite the 
development and distribution of a manual describing the 
processes used to collect and analyse the cost data of 
the first trial (31). To facilitate a comparative costing 
and to enhance standardisation across our trials, the UK 
partner institution appointed an economics team in 2009 

Table 2. Costing tool structure 



with overall responsibility for the evaluation of the trials. 
This led to the design of a common costing tool to be 
used for each site. 

Because the software platform for the costing tool 
needed to be universally accessible, we avoided the use of 
specialised statistical and data management software. We 
used a compatible version of MS Excel, already available 
and extensively used by all partner institutions. The tool 
was designed in a way to be easy for a non-economist to 
understand and use. To ensure that this, the costing 
contact and other key researchers at the trial site met for 
a one-day workshop to discuss the purpose of the tool, to 
learn how to use it, and to agree on definitions of relevant 
cost categories. Following a first draft of the tool, the UK 
economics team conducted a series of one-to-one training 
sessions with key costing contacts at each site. These 
sessions revealed the degree to which flexibility was 
needed in the tool to incorporate previously described 
differences in the structure of the trial, activities, and 
priorities among the sites. Feedback, opinions, and con- 
cerns regarding the use of the tool continue to be shared 
between the key costing personnel in the UK and the trial 
sites via e-mail and Skype. 

The costing tool will be available in the public domain 
when the trial data are released. Table 2 presents the 



Categories of costs 

Main input categories: 
Staff time and salaries 

Materials and equipment 



Capital costs 



Joint costs 



Summary sheets and additional data 
Women's group facilitation 

Health service strengthening or 

other intervention(s) 

Monitoring and evaluation 

Process evaluation 

Research 

Results sheets: 
National currency US dollars 
Cost-effectiveness 



Data input 

Salaries and other staff costs. Annual 
allocation (hours or percentage). 
Purchase year and price. 
Allocation. 

Purchase date, price, and expected useful 
life (for items owned). 
Annual recurrent costs. 
Annual allocation. 

Summarises share of staff, material, and 
capital costs already allocated to joint costs. 
Allows additional items to be input. 

entry by programme component: 

Each sheet summarises the share of 
allocated joint costs and directly allocated 
staff, material, and capital costs. Additional 
costs may also be entered. 



Detail of costs with various breakdowns. 
The user is able to input programme effect 
data and calculate cost-effectiveness. 



Examples of items 

Salaries of programme staff; facilitators; 
identifiers; volunteers. 
Items that have no resale value after 1 
year (e.g. picture cards, questionnaires, 
HIV testing kits, raincoats). 
Buildings, vehicles and equipment with 
an expected useful life >1 year (e.g. 
laptops, bicycles, cars, photocopiers, 
office premises). 

Audit fees, licence fees, fuel and travel 
refunds, training and recruitment costs, 
meeting costs. 

Fuel and travel refunds, training and 
recruitment costs, project material 
development costs, meeting costs. 
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structure of the tool. The main worksheets for entering 
data are staff, material, capital, and joint costs. We also 
created worksheets that summarise the costs (including 
a share of joint costs) by programme components (e.g. 
women's groups) and that allow additional items to be 
entered. The final worksheets present results and allow 
effect data to be entered and cost-effectiveness results to 
be calculated. 

Discussion 

Some of the challenges that we faced while collecting 
and analysing the cost data across multiple sites may be 
common to data collection processes for single and mul- 
tiple site costings. For both, we offer practical solutions 
for resource poor settings. We do not discuss analytical 
methods in detail because the use of decision trees 
or modelling techniques is more closely related to the 
purpose of economic evaluation. 

An important consideration in a costing is the accurate 
identification and categorisation of different costs. In our 
trials, this was the case with identifying start-up costs, 
in particular. Early start-up activities were not always 
separately identified or even included in project accounts. 
This is likely to be an issue with all trials where there 
are multiple funding sources and where interventions 
are planned before funding begins. We recommend that 
in such a scenario, the staff involved build a common 
understanding of cost components and activities and 
work to reconcile cost data from multiple sources that 
may be duplicated in different databases or that are 
available in different formats. 

Another very important consideration is whether to 
exclude research costs from analysis. Separating out the 
research costs specific to a trial setting from intervention 
implementation costs is informative if the results are used 
to estimate scale-up or implementation costs in another 
setting. We excluded research costs from our analysis 
but included monitoring and evaluation costs. Although 
research activities are distinctly necessary in a trial 
setting, some degree of monitoring and evaluation is 
desirable when interventions are rolled out. One solution 
for distinguishing between the two is to identify a mini- 
mum amount of cost for monitoring and evaluation and 
to assign the remainder to research. Another is to identify 
activities that are clearly research driven, such as control 
area surveillance, and allocate costs for those activities 
to research. In the absence of a good allocation method, 
a conservative approach such as ours is advisable. We 
simply reported all monitoring and evaluation costs as 
part of the total project cost. In our trials, we could not 
rule out the possibility that intense and visible surveil- 
lance activities in the community have positive external- 
ities or other unintended positive health outcomes. 
Excluding monitoring and evaluation in this situation 



would risk underestimating the total cost and over- 
estimating the cost-effectiveness of the intervention. 

To carry out a comparative CEA across multiple sites, 
it is imperative to collect data that is consistent across all 
sites. In our experience, collaboration with key costing 
personnel at each site, led by the UK economics team, 
proved to be very successful in identifying and collecting 
provider cost data. The design and use of a custom cost 
collection tool built on an accessible platform allowed 
data to be collected in a consistent manner. The tool also 
allowed a degree of flexibility in the data collection that 
reflected important site-specific characteristics. The tool 
and the manner in which it was used by the in-country 
and UK partners allowed knowledge sharing and capa- 
city building. Further, the collaborative process allowed 
us to collect the data in a timely manner and to ensure 
quality. 

We found no consensus in the guidelines on how to 
choose appropriate time horizons or discount rates for 
costs, especially for community participatory trials in 
low- and middle-income countries. We believe that the 
choice of these would depend on the setting of the trial as 
well as the expectations of scale-ups or replication. 

Learning from our experience over a decade, we are 
able to contribute toward the improvement of cost data 
collection methodology and systems. This experience has 
highlighted the importance of planning ahead and of 
being prepared for the possibility of unexpected events. 
For example, the start-up of the Dhanusha and Mchinji 
trials was delayed because of a Maoist uprising and 
because of unexpected personal events, respectively. 
Guidelines or recommendations on whether and how to 
factor in such events into analysis are non-existent. In our 
case, the delays occurred before implementation. The 
associated costs were therefore included as part of the 
start-up costs, effectively becoming one of many factors 
that contributed to differences in start-up costs across the 
sites. 

Finally, it is worth noting that, given the lifespan of 
these trials, the staff involved in evaluating the costs and 
benefits of the interventions has changed. We tried to 
ensure consistency over time by exchanging information 
with outgoing researchers, constructing a costing tool 
that is standardised yet has sufficient flexibility to be 
adapted by partners for subsequent use, and by training 
groups of staff on the methods used. The truth is that, 
with the movement of researchers into and from a pro- 
ject, there will also be a movement of ideas. This presents 
an opportunity to improve what has been done in the 
past and poses a challenge to maintain consistency and 
comparability over time. Trials conducted over multiple 
sites tend to be large collaborations of implementing and 
research partners. Each partner will bring their own ideas 
about what is most important within the research process 
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and how the objectives of the trial may best be achieved. 
Many of the decisions made regarding a comparative 
CEA will be made by these different teams. These deci- 
sions are likely to reflect necessary compromise in the 
design, implementation, and analysis. 

Our reflections on the practical considerations of 
collecting cost data for a comparative costing of multiple 
sites is not an exhaustive list of guidelines or of the chal- 
lenges faced. It outlines some of the main steps involved 
in data collection for the economic evaluation of a single 
site trial and describes our experience putting these steps 
into practice across multiple sites. In doing so, we provide 
pragmatic solutions to common problems that may be 
faced when costing a cluster randomised, controlled, trial 
in resource poor settings. Further, we hope that the dis- 
cussion in this paper encourages researchers other than 
economists to conduct economic evaluations of their 
interventions. 
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